Quantifying the behavior of stock correlations under market stress

Understanding correlations in complex systems is crucial in the face of turbulence, such as the ongoing financial crisis. However, in complex systems, such as financial systems, correlations are not constant but instead vary in time. Here we address the question of quantifying state-dependent correlations in stock markets. Reliable estimates of correlations are absolutely necessary to protect a portfolio. We analyze 72 years of daily closing prices of the 30 stocks forming the Dow Jones Industrial Average (DJIA). We find the striking result that the average correlation among these stocks scales linearly with market stress reflected by normalized DJIA index returns on various time scales. Consequently, the diversification effect which should protect a portfolio melts away in times of market losses, just when it would most urgently be needed. Our empirical analysis is consistent with the interesting possibility that one could anticipate diversification breakdowns, guiding the design of protected portfolios

Bayesian networks in survey data: Robustness and sensitivity issues

Bayesian networks (BN) implement a graphical model structure known as a directed acyclic graph (DAG) that is popular in statistics, machine learning, and artificial intelligence. They enable an effective representation and computation of a joint probability distribution (JPD) over a set of random variables. The paper focuses on the selection of a robust network structure according to different learning algorithms and the measure of arc strength using resampling techniques. Moreover, it shows how 'what-if' sensitivity scenarios are generated with BN using hard and soft evidence in the framework of predictive inference. Establishing a robust network structure and using it for decision support are two essential enablers for efficient and effective applications of BN to improvements of products and processes. A customer-satisfaction survey example is presented and R scripts are provided

An Exact Formula for the Average Run Length to False Alarm of the Generalized Shiryaev-Roberts Procedure for Change-Point Detection under Exponential Observations

We derive analytically an exact closed-form formula for the standard minimax Average Run Length (ARL) to false alarm delivered by the Generalized Shiryaev-Roberts (GSR) change-point detection procedure devised to detect a shift in the baseline mean of a sequence of independent exponentially distributed observations. Specifically, the formula is found through direct solution of the respective integral (renewal) equation, and is a general result in that the GSR procedure's headstart is not restricted to a bounded range, nor is there a "ceiling" value for the detection threshold. Apart from the theoretical significance (in change-point detection, exact closed-form performance formulae are typically either difficult or impossible to get, especially for the GSR procedure), the obtained formula is also useful to a practitioner: in cases of practical interest, the formula is a function linear in both the detection threshold and the headstart, and, therefore, the ARL to false alarm of the GSR procedure can be easily computed.Comment: 9 pages; Accepted for publication in Proceedings of the 12-th German-Polish Workshop on Stochastic Models, Statistics and Their Application

Challenges in network science: Applications to infrastructures, climate, social systems and economics

Network theory has become one of the most visible theoretical frameworks that can be applied to the description, analysis, understanding, design and repair of multi-level complex systems. Complex networks occur everywhere, in man-made and human social systems, in organic and inorganic matter, from nano to macro scales, and in natural and anthropogenic structures. New applications are developed at an ever-increasing rate and the promise for future growth is high, since increasingly we interact with one another within these vital and complex environments. Despite all the great successes of this field, crucial aspects of multi-level complex systems have been largely ignored. Important challenges of network science are to take into account many of these missing realistic features such as strong coupling between networks (networks are not isolated), the dynamics of networks (networks are not static), interrelationships between structure, dynamics and function of networks, interdependencies in given networks (and other classes of links, including different signs of interactions), and spatial properties (including geographical aspects) of networks. This aim of this paper is to introduce and discuss the challenges that future network science needs to address, and how different disciplines will be accordingly affected. Graphical abstrac

Challenges in network science: Applications to infrastructures, climate, social systems and economics

Network theory has become one of the most visible theoretical frameworks that can be applied to the description, analysis, understanding, design and repair of multi-level complex systems. Complex networks occur everywhere, in man-made and human social systems, in organic and inorganic matter, from nano to macro scales, and in natural and anthropogenic structures. New applications are developed at an ever-increasing rate and the promise for future growth is high, since increasingly we interact with one another within these vital and complex environments. Despite all the great successes of this field, crucial aspects of multi-level complex systems have been largely ignored. Important challenges of network science are to take into account many of these missing realistic features such as strong coupling between networks (networks are not isolated), the dynamics of networks (networks are not static), interrelationships between structure, dynamics and function of networks, interdependencies in given networks (and other classes of links, including different signs of interactions), and spatial properties (including geographical aspects) of networks. This aim of this paper is to introduce and discuss the challenges that future network science needs to address, and how different disciplines will be accordingly affected

Index Cohesive Force Analysis Reveals That the US Market Became Prone to Systemic Collapses Since 2002

BACKGROUND: The 2007-2009 financial crisis, and its fallout, has strongly emphasized the need to define new ways and measures to study and assess the stock market dynamics. METHODOLOGY/PRINCIPAL FINDINGS: The S&P500 dynamics during 4/1999-4/2010 is investigated in terms of the index cohesive force (ICF--the balance between the stock correlations and the partial correlations after subtraction of the index contribution), and the Eigenvalue entropy of the stock correlation matrices. We found a rapid market transition at the end of 2001 from a flexible state of low ICF into a stiff (nonflexible) state of high ICF that is prone to market systemic collapses. The stiff state is also marked by strong effect of the market index on the stock-stock correlations as well as bursts of high stock correlations reminiscence of epileptic brain activity. CONCLUSIONS/SIGNIFICANCE: The market dynamical states, stability and transition between economic states was studies using new quantitative measures. Doing so shed new light on the origin and nature of the current crisis. The new approach is likely to be applicable to other classes of complex systems from gene networks to the human brain

Local variation of hashtag spike trains and popularity in Twitter

We draw a parallel between hashtag time series and neuron spike trains. In each case, the process presents complex dynamic patterns including temporal correlations, burstiness, and all other types of nonstationarity. We propose the adoption of the so-called local variation in order to uncover salient dynamics, while properly detrending for the time-dependent features of a signal. The methodology is tested on both real and randomized hashtag spike trains, and identifies that popular hashtags present regular and so less bursty behavior, suggesting its potential use for predicting online popularity in social media.Comment: 7 pages, 7 figure

Statistically validated networks in bipartite complex systems

Many complex systems present an intrinsic bipartite nature and are often described and modeled in terms of networks [1-5]. Examples include movies and actors [1, 2, 4], authors and scientific papers [6-9], email accounts and emails [10], plants and animals that pollinate them [11, 12]. Bipartite networks are often very heterogeneous in the number of relationships that the elements of one set establish with the elements of the other set. When one constructs a projected network with nodes from only one set, the system heterogeneity makes it very difficult to identify preferential links between the elements. Here we introduce an unsupervised method to statistically validate each link of the projected network against a null hypothesis taking into account the heterogeneity of the system. We apply our method to three different systems, namely the set of clusters of orthologous genes (COG) in completely sequenced genomes [13, 14], a set of daily returns of 500 US financial stocks, and the set of world movies of the IMDb database [15]. In all these systems, both different in size and level of heterogeneity, we find that our method is able to detect network structures which are informative about the system and are not simply expression of its heterogeneity. Specifically, our method (i) identifies the preferential relationships between the elements, (ii) naturally highlights the clustered structure of investigated systems, and (iii) allows to classify links according to the type of statistically validated relationships between the connected nodes.Comment: Main text: 13 pages, 3 figures, and 1 Table. Supplementary information: 15 pages, 3 figures, and 2 Table

Financial time series prediction using spiking neural networks

In this paper a novel application of a particular type of spiking neural network, a Polychronous Spiking Network, was used for financial time series prediction. It is argued that the inherent temporal capabilities of this type of network are suited to non-stationary data such as this. The performance of the spiking neural network was benchmarked against three systems: two "traditional", rate-encoded, neural networks; a Multi-Layer Perceptron neural network and a Dynamic Ridge Polynomial neural network, and a standard Linear Predictor Coefficients model. For this comparison three non-stationary and noisy time series were used: IBM stock data; US/Euro exchange rate data, and the price of Brent crude oil. The experiments demonstrated favourable prediction results for the Spiking Neural Network in terms of Annualised Return and prediction error for 5-Step ahead predictions. These results were also supported by other relevant metrics such as Maximum Drawdown and Signal-To-Noise ratio. This work demonstrated the applicability of the Polychronous Spiking Network to financial data forecasting and this in turn indicates the potential of using such networks over traditional systems in difficult to manage non-stationary environments. © 2014 Reid et al

Challenges in Complex Systems Science

FuturICT foundations are social science, complex systems science, and ICT. The main concerns and challenges in the science of complex systems in the context of FuturICT are laid out in this paper with special emphasis on the Complex Systems route to Social Sciences. This include complex systems having: many heterogeneous interacting parts; multiple scales; complicated transition laws; unexpected or unpredicted emergence; sensitive dependence on initial conditions; path-dependent dynamics; networked hierarchical connectivities; interaction of autonomous agents; self-organisation; non-equilibrium dynamics; combinatorial explosion; adaptivity to changing environments; co-evolving subsystems; ill-defined boundaries; and multilevel dynamics. In this context, science is seen as the process of abstracting the dynamics of systems from data. This presents many challenges including: data gathering by large-scale experiment, participatory sensing and social computation, managing huge distributed dynamic and heterogeneous databases; moving from data to dynamical models, going beyond correlations to cause-effect relationships, understanding the relationship between simple and comprehensive models with appropriate choices of variables, ensemble modeling and data assimilation, modeling systems of systems of systems with many levels between micro and macro; and formulating new approaches to prediction, forecasting, and risk, especially in systems that can reflect on and change their behaviour in response to predictions, and systems whose apparently predictable behaviour is disrupted by apparently unpredictable rare or extreme events. These challenges are part of the FuturICT agenda